Improving the Prediction Accuracy of Echo State Neural Networks by Anti-Oja's Learning

Echo state neural networks, which are a special case of recurrent neural networks, are studied from the viewpoint of their learning ability, with a goal to achieve their greater prediction ability. A standard training of these neural networks uses pseudoinverse matrix for one-step learning of weights from hidden to output neurons. This regular adaptation of Echo State neural networks was optimized by updating the weights of the dynamic reservoir with Anti-Oja’s learning. Echo State neural networks use dynamics of this massive and randomly initialized dynamic reservoir to extract interesting properties of incoming sequences. This approach was tested in laser fluctuations and MackeyGlass time series prediction. The prediction error achieved by this approach was substantially smaller in comparison with prediction error achieved by a standard algorithm.